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PREFACE 


This  report  was  created  for  the  F-16  Aircrew  Training  De¬ 
velopment  Project  contract  no.  F02604-79-C8875  for  the  Tactical 
Air  Command  to  comply  with  the  requirements  of  CDRL  no.  S041. 

The  project  entailed  the  design  and  development  of  an  instruc¬ 
tional  system  for  the  F-16  RTU  and  instructor  pilots.  During  the 
course  of  the  project,  a  series  of  development  reports  was  issued 
describing  processes  and  products.  A  list  of  those  reports 
follows  this  page.  The  user  is  referred  to  Report  No.  34,  A 
Users  Guide  to  the  F-16  Training  Development  Reports,  for  an 
overview  and  explanation  of  the  series,  and  Report  No.  35,  F-16 
Final  Report,  for  an  overview  of  the  Instructional  System  De¬ 
velopment  Project. 
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EXECUTIVE  SUMMARY 


\ 

Because  the  training  development  contractor  has  been 
involved  early  in  the  life  cycle  of  the  aircraft,  the  F-16  pro- 
ject  provides  an  opportunity  for  instructional  developers  to 
provide  input  to  the  design  of  training  devices  to  be  developed 
for  the  F-16  training  system.  Some  of  the  capabilities  already 
incorporated  in  the  design  of  the  F-16  Operational  Flight  Trainer 
(OFT)  such  as  freeze,  performance  replays,  and  automated  measure¬ 
ment  are  highly  desirable  instructional  features.  In  addition, 
it  is  recommended  that  capabilities  be  provided  by  grade  sheet 
production,  simulator  setup  and  off-line  debriefings,  mission 
status  and  look-ahead,  a  self-instructional  capability,  and  a 
"help"  capability  for  instructors.  Two  problems  with  the  physi¬ 
cal  arrangement  of  the  OFT  for  training  demonstrations  were  iden¬ 
tified.  A  series  of  principles  were  specified  for  the  formatting 
of  instructional  displays.  A  number  of  recommendations  were  made 
regarding  the  design  of  the  OFT  for  simulating  aircraft  malfunc¬ 
tions.  Finally,  recommendations  were  made  in  terms  of  designing 
the  OFT  to  accomodate  growth  and  changes  over  the  F-16  training 
system  lifespan. 

While  the  involvement  of  the  ISD  team  in  the  simulator 
design  process  has  been  helpful  in  identifying  design  deficien¬ 
cies,  participation  prior  to  simulator  procurement  in  future 
projects  would  result  in  training  devices  which  are  more  closely 
aligned  to  the  requirements  of  the  training  system. 
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RECOMMENDATIONS 

FOR  F-16  OPERATIONAL  FLIGHT  TRAINER  (OFT) 
DESIGN  IMPROVEMENTS 


1.0  INTRODUCTION 


The  F-16  Aircrew  Training  Development  Project  provides  a 
unique  opportunity  for  instructional  developers  to  become 
involved  in  design  of  an  aircrew  training  simulator  for  opera¬ 
tional  flight  training.  The  long  lead  times  required  for  the 
development  of  complex  operational  flight  trainers  (OFTs)  and  the 
customary  late  arrival  of  instructional  developers  upon  the  scene 
in  an  aircrew  training  system  development  usually  prevents 
instructional  scientists  from  applying  the  principles  of  instruc¬ 
tional  systems  development  (ISD)  to  such  designs.  As  a  result, 
the  dominant  practice  has  been  to  design  simulators  to  meet 
standards  of  simulation  fidelity  rather  than  standards  based  upon 
training  requirements. 

In  contrast  to  this  normal  state  of  affairs,  the  F-16 
aircrew  training  development  contractor  has  been  involved  earlier 
in  the  life  cycle  of  the  aircraft  system,  at  a  stage  when  many 
details  of  the  F-16  OFT  design  are  not  yet  firm.  The  unusual 
opportunity  is  presented  to  align,  to  the  extent  possible, 
simulator  design  characteristics  (both  physical  and  software) 
with  the  requirements  of  the  instructional  system  developed  for 
training  F-16  pilots  and  instructor  pilots. 

The  purpose  of  this  paper  is  to  recommend  design  features 
for  the  F-16  OFT  which  support  training  requirements  of  the  F-16 
instructional  system.  The  cost/benefit  tradeoffs  prerequisite  to 
the  implementation  of  the  suggested  features  have  not  been 
conducted.  The  suggestions  presented  here  are  based  upon  the  ISD 
program  design  products,  preliminary  descriptions  of  the  simula¬ 
tor  design  concept  provided  by  the  simulator  contractor,  and 
suggestions  from  the  F-16  OTD  team  and  other  Tactical  Air  Command 
(TAC)  agencies  involved  in  simulator  design,  review  and  procure¬ 
ment.  Proposals  contained  herein  have  been  developed  through 
preliminary  design  review  meetings  at  the  simulator  contractor's 
facility,  and  through  examination  of  fragmentary  reports  and 
preliminary  specifications  provided  at  that  meeting.  No  system¬ 
atic  review  of  the  completed  design  has  been  accomplished,  and  to 
the  best  of  our  knowledge,  no  thorough  analytical  determination 
of  functional  requirements  for  the  F-16  OFT/WST  (weapon  system 
trainer)  was  undertaken  prior  to  the  commencement  of  design. 
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While  the  involvement  of  the  ISD  team  in  the  simulator 
design  review  process  has  been  helpful  in  identifying  selected 
design  deficiencies  in  relation  to  the  functional  objectives  of 
the  OFT,  participation  prior  to  simulator  procurement  specifica¬ 
tion  in  future  projects  will  result  in  a  more  extensive  and 
influential  statement  of  design  objectives.  Ideally,  the  ISD 
effort  should  produce  a  set  of  training-task-based  performance 
specifications  which,  taken  together  with  the  simulator  concept 
of  employment,  including  instructional  strategies,  would  form  the 
nucleus  of  future  procurement  specifications. 

This  fact  notwithstanding,  an  attempt  has  been  made  to  make 
recommendations  in  areas  of  the  simulator  design  which  directly 
relate  to  the  application  of  instructional  strategy  or  the  opera¬ 
tion  of  the  simulator  as  a  tool  by  simulator  instructor  pilots. 

It  is  the  philosophy  of  ISD  that  the  simulator  is  an  instruc¬ 
tional  tool  to  be  used  by  a  training  manager  or  an  instructor  for 
the  purpose  of  meeting  specific  student  needs  as  the  student  pro¬ 
gresses  toward  the  acquisition  of  identified  psychomotor  skills. 
Just  as  any  tool  meant  to  be  used  by  a  workman  should  be  designed 
to  meet  the  workman's  need,  the  simulator  should  be  designed  for 
use  by  the  instructor  as  an  agent  in  producing  specific 
behavioral  changes  in  the  student. 

It  is  the  principal  concern  of  this  report  to  advocate  the 
,  application  of  ISD  principles  in  the  simulator  design  and  to 

I  ensure  that  a  proper  perspective  of  the  OFT  as  a  training  tool 

guides  the  design  process. 

Simulators  are  vastly  complex  devices,  and  to  fully  evaluate 
;  simulator  designs  requires  breadth  and  depth  of  knowledge  in 

several  specialty  areas.  Certain  aspects  of  the  F-16  simulator 
design,  particularly  engineering  features,  are  beyond  competent 
comment  by  the  authors.  In  general  the  recommendations  do  not 
extend  to  detailed  hardware  engineering  that  gives  the  machine 
its  capabilities. 

To  organize  the  contents  of  this  paper,  recommendations  are 
given  under  three  main  headings,  dividing  the  study  into  sections 
on  (1)  instructor  operator  station  interface  design,  (2)  specific 
instructional  capabilities,  and  (3)  design  for  growth.  Within 
\  each  of  these  areas  problems  in  the  simulator  design  as  presently 

conceived  are  identified,  and  recommendations  are  made  aimed  at 
solution. 
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2.0  SPECIFIC  INSTRUCTIONAL  CAPABILITIES 


Characteristics  of  a  simulator  that  transform  it  from  a 
high-fidelity  aircraft  system  mimic  into  a  training  device  are 
its  instructional  features.  These  features  allow  the  instructor 
to  produce  simulation  exercises  with  the  most  appropriate  blend 
of  pauses,  feedback,  cueing,  instructor-student  interactions, 
instruction,  and  motivation. 

Several  useful  instructional  features  are  already  incorpo¬ 
rated  into  the  F-16  OFT  design  including  capabilities  for  freeze, 
replay  of  student  performance,  demonstrations,  and  automated 
performance  measurement  for  simple  performances. 

This  section  discusses  specific  additional  capabilities 
which  will  facilitate  the  instructor's  role  as  a  manager  of 
simulator  exercises. 


2.1  Grade  Sheet  Production 

Problem:  The  requirements  for  data  for  performance  measure¬ 

ment  have  prompted  a  move  in  simulator  design  toward  production 
of  appropriate  data  outputs.  The  approach  to  creating  these 
outputs,  however,  has  often  been  to  produce  large  volumes  of  data 
in  the  hope  that  within  the  data  the  instructor  will  be  able  to 
find  some  measures  of  personal  usefulness.  The  resulting  data 
outputs  are  generally  excessively  bulky  and  difficult  for  the 
instructor  to  use  in  a  debriefing  session  immediately  following 
the  flight.  Moreover,  the  largeness  of  the  body  of  data  produced 
encourages  variations  in  interpretations  of  variables  by  differ¬ 
ent  instructors.  The  absence  of  focus  appears  to  encourage 
non-standardization  of  evaluation. 

Recommendation :  The  F-16  OFT  should  produce  carefully 

structured  records  of  relevant  data  output  in  a  format  readily 
usable  by  instructors  for  debriefing  and  historical  purposes. 

The  emphasis  in  this  recommendation  is  on  the  production  of 
documentation  from  the  simulator  which  can  be  effectively  used  by 
the  instructor  during  the  debriefing  process,  and  which  contains 
sufficient  information  that  it  be  usable  as  a  part  of  the  stu¬ 
dent's  grade  record.  An  example  of  such  a  printout  is  presented 
in  Figure  1.  That  figure  represents  a  printout  page  as  it  might 
appear  at  the  end  of  the  takeoff  and  departure  portion  of  a 
navigation  exercise.  The  novel  feature  which  distinguishes  this 
page  is  that  the  end  state  of  the  mission  segment  is  presented  as 
well  as  the  critical  parameter  values  associated  with  the 
airplane's  flight  at  earlier  significant  points  in  the  mission. 
Comparison  between  these  values  and  criterion  values  can  be  made 
easily  for  each  point  during  the  mission.  The  generation  of 
these  displays  requires  the  simulator  to  freeze  the  desired  data 
on  aircraft  flight  performance  on  the  page  at  appropriate 
locations  as  the  flight  progresses. 
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Figure  1  —  A  Sample  Printout  Following  Simulator  Use  Suitable  for 
Debriefing  and  Grading 


It  is  believed  that  useful  grade  sheets  could  be  produced 
for  virtually  all  of  the  major  training  sequences  carried  out  in 
the  simulator,  including  air-to-air  combat  in  advanced  versions. 
Much  thought  and  preparation,  however,  must  go  into  the  design 
both  of  the  machines  and  of  the  printout  formats  to  accomplish 
this  objective. 


2.2  Offline  Problem  Set-up/Debrief 

Problem:  Simulator  instructional  sessions  are  typically 

scheduled  to  last  from  1-1/2  to  2  hours  including  time  for  simu¬ 
lator  programming  and  student  debriefing.  If  simulator  program¬ 
ming  and  post-flight  review  become  lengthy,  the  percentage  of  the 
scheduled  session  remaining  for  the  actual  simulated  flight 
rapidly  diminishes  with  consequent  reduction  in  the  time  effi¬ 
ciency  of  simulator  training.  For  example,  if  set-up  takes 
15  minutes  of  interaction  with  the  instructor/operator  station 
(IOS)  console,  and  debrief  involves  15  minutes  of  replay  and 
discussion  with  the  student,  only  2/3  of  the  scheduled  1-1/2  hour 
period  can  be  available  for  flight  simulation.  This  level  of 
utilization  equates  to  a  50%  increase  in  real  simulator  training 
time  cost.  Means  must  be  sought  to  achieve  the  highest  possible 
level  of  simulator  utilization. 


rent  with  flight  simulation  operations,  the  utilization  potential 
and  operating  costs  per  student  training  hour  can  be  immediately 
improved.  For  mission  set-up,  an  IOS  "emulator"  could  be  devel¬ 
oped.  This  device  might  be  a  micro-computer  based  dynamic 
replica  of  a  single  IOS  display  station  capable  of  presenting 
simulated  "menus"  of  instructor  options  exactly  as  they  appear  on 
the  IOS.  Instructor  selections  on  this  IOS  "emulator"  would  be 
either  relayed  into  a  buffer  store  in  the  simulator  memory  for 
retrieval  at  the  start  of  the  succeeding  training  period,  or 
recorded  onto  some  transportable  media  for  physical  relocation  to 
and  transcription  by  the  simulator  at  that  same  time. 

Off-line  reconstruction  of  simulated  flight  events  would 
involve  somewhat  more  complex  and  expensive  equipment;  thus  a 
careful  determination  must  be  made  of  the  cost-effectiveness  of 
this  proposed  feature.  The  potential  benefits  of  an  off-line 
debriefing  capability  are  significant,  however.  Anticipated 
benefits  of  this  arrangement  are:  (1)  relief  in  the  demand  for 
the  IOS  at  critical  transition  periods,  (2)  lower  operating  cost, 
(3)  further  offloaded  demand  for  the  IOS  if  the  debrief  capabil¬ 
ity  is  designed  to  incorporate  snapshot  production,  (4)  automated 
performance  analysis  capabilities,  and  (5)  elimination  of  time 
stress  that  often  detrimentally  influences  the  review  process. 

It  is  elsewhere  suggested  that  consideration  be  given  to 
designing  air  combat  manuevering  range  ( ACMR)  replay  compati- 


Recommendat ion :  A  capability  should  be.  established  to  per¬ 
form  simulator  set-up  and/or  student  debriefings  off-line.  If  a 
means  can  be  found  to  perform  set-up  and  debriefing  tasks  concur 
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bility  into  the  F-16  OFT.  (See  Section  4.2  of  this  report.)  It 
would  be  desirable  if  this  feature  were  incorporated  in  any 
debriefing  device  like  the  one  described  above.  Instructors 
would  have  more  frequent  opportunities  to  use  the  simulator 
equipment  in  this  event,  which  would  equate  to  greater  confidence 
and  skill  in  utilization  of  the  simulator  equipment  itself.  This 
feature  would  also  help  to  amortize  costs  over  a  greater  usage 
base  and  thus  lower  operating  expense  still  further. 


2.3  Mission  Status  and  Look  Ahead 

Problem:  During  the  course  of  a  simulation  exercise, 

planned  and  unplanned  events  occur  in  which  the  instructor  must 
participate,  either  as  a  source  of  information  for  the  student, 
or  as  a  problem  controller  by  interacting  with  the  IOS  controls 
and  displays.  The  instructor  must  not  only  be  fully  aware  of 
events  which  have  already  transpired  in  an  exercise  and  student 
performance  during  each,  but  must  be  able  to  look  ahead  and 
anticipate  coming  events,  both  planned  and  random  so  that  he  may 
carry  out  his  responsibilities.  The  demands  on  an  instructor's 
time  while  managing  a  simulation  exercise  are  great,  and  the 
requirement  to  anticipate  upcoming  events  while  managing  ongoing 
ones  is  an  added  burden  which  can  be  avoided.  To  alleviate  the 
memory  burden  for  instructors,  and  to  make  preparation  for  both 
expected  and  random  events  less  demanding  of  attention,  a  cueing 
mechanism  for  alerting  the  instructor  of  upcoming  events  will  be 
helpful. 

Recommendation:  It  is  recommended  that  a  feature  be  pro¬ 

vided  allowing  the  instructor  pilot  (IP)  to  examine  the  profile 
of  a  training  mission  (past  and  future)  while  the  mission  is 
being  conducted,  and  determine  the  progress  through  the  mission 
to  that  point  while  being  alerted  to  the  nature  and  time  of 
occurrence  of  events  yet  to  transpire  during  the  mission.  This 
feature  may  take  one  of  two  forms.  Either  it  may  be  a  feature 
separate  from  the  problem  pages,  or  it  may  be  implemented  as  a 
simple  overlay  imposed  upon  the  screen  while  an  instructor  is  yet 
monitoring  a  given  training  problem  in  the  form  of  leading  logic 
for  instructor  actions.  The  look-ahead  capability  should  also 
assist  the  IP  in  avoiding  violation  of  the  simulator  operational 
parameter  range  limits  by  indicating  threats  that  could  terminate 
the  mission,  by  warning  the  IP  of  pending  automatically  inserted 
malfunctions  (Section  3.6),  and  by  alerting  the  IP  of  upcoming 
responsibilities,  including  radio  calls  and  clearances.  In  this 
connection,  it  is  suggested  that  such  leading  logic  take  the  form 
of  a  flashing  message  on  the  IOS  displays  requiring  an  acknow¬ 
ledge  button  activation  to  stop  the  flashing.  This  status  and 
look  ahead  capability  could  also  monitor  and  alert  the  IP  to  the 
possibility  of  selection  of  incompatible  or  catastrophic  combina¬ 
tions  of  parameters. 
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3.0  INSTROCTOR/OPERATOR  STATION  INTERFACE  DESIGN 


The  F-16  OFT  design  must  insure  that  the  physical  layout  and 
aechanical  and  functional  operation  requirements  of  the  simulator 
do  not  interfere  with  its  use  as  a  training  tool.  This  section 
contains  recommendations  for  design  improvements  to  promote  an 
instructor-job  compatible  OFT. 


3.1  Self  Instructional  Feature 

Problems:  There  is  a  need  for  training  OFT  instructors  to 
use  the  siaulator.  Two  problems  exist. 

1.  The  flexibility  provided  by  modern  flight  simulators 
requires  that  the  operator  make  many  choices  in  selec¬ 
ting  desired  operating  conditions.  Mastery  of  simulator 
control  implies  training  in  option  selection  and  opera¬ 
tion  of  the  siaulator  interface*  which  quickly  becomes 
more  than  a  trivial  training  problem. 

2.  Dedicated  siaulator  operation  personnel  are  not  employ¬ 
ed*  and  training*  though  often  implemented  as  part  of 
the  qualification  for  instructors,  is  not  maximally 
effective.  USAF  simulators  use  frequently-rotated 
personnel  as  instructors*  who  in  addition  are  often 
assigned  additional  duties  and  instruct  on  the  siaulator 
equipment  infrequently.  In  this  situation*  extensive 
training  is  impractical  and  not  likely  to  be  well- 
retained.  Consequently  simulators  are  not  used  to  full 
efficiency  or  capability. 

Recommendation  fl:  The  simulator  should  incorporate  a  self- 
instructional  feature  for  IPs  that  operates  through  on-sinulator 
tutor ials.  A  self- instructional  capability  could  be  produced  to 
be  implemented  either  at  the  main  instructor  console  or  at  an 
isolated  console  (Section  2.2)  while  the  siaulator  is  running 
instructional  problems.  The  memory  and  central  processing  unit 
(CPU)  load  required  by  instructional  delivery  by  computer  is 
relatively  low*  and  it  is  highly  likely  that  sufficient  tine  will 
exist  within  the  operating  system  of  the  simulator  to  service  an 
instructional  program  at  the  same  time  a  simulation  problem  is 
being  run.  This  will  be  especially  true  if  other  recommendations 
of  this  report  concerning  reserve  computational  power  are  imple¬ 
mented.  (See  Section  4.6.) 

Recommendation  #2:  It  is  recommended  that  a  "HELP*  function 
be  available  to  IPs  while~~using  the  simulator  for  the  purpose  of 
guiding  or  suggesting  instructor  act lonsA>Pt ions  at  points  where 
the  instructor  recognizes  that  he  does  not  understand  what  ~~~ 
options  are  available!  This  JHELP*  function  could  summon  to  the 
display  specific  computer  recommendations  for  control  action 
associated  with  the  current  state  of  the  simulator*  or  at  very 


7 


1 


least  a  map  or  menu  of  options  open  to  the  instructor  at  a  given 
time  . 


3.2  Mockup  Physical  Arrangement  (I) 

Problem:  Since  the  operators  will  be  looking  up  at  the 

midpoint  of  the  cathode  ray  tube  (CRT)  displays,  and  since  the 
front  surface  of  such  displays  typically  reflect  specularly  with 
high  efficiency,  the  potential  for  nuisance  reflection  of  room 
lights  above  and  behind  the  operator  is  quite  high.  While  the 
annoyance  created  by  these  reflections  could  be  eliminated  by 
careful  placement  of  the  room  lighting,  such  a  constraint  is 
inconsistent  with  USAF  facilities  plans. 


Recommendation :  Two  minor  modifications  to  the  IQS  will  be 

helpful  in  minimizing  the  problem.  First,  consideration  should 
be  given  to  tilting  the  CRT's  forward.  Such  an  inclination  would 
lessen  the  probability  that  specular  reflections  of  the  overhead 
lights  could  reach  the  operator's  eye.  A  second  solution  would 
be  to  use  either  neutral  density  or  matched  spectral  filtration 
on  the  CRT  faceplates.  Filtration  would  attenuate  reflected  room 
light  by  a  factor  equal  to  the  square  of  the  filter  transmission 
percentage,  because  the  incident  light  would  have  to  pass  through 
the  filter  twice.  Light  from  the  CRT  phosphor  would  pass  through 
the  filter  only  once,  and  although  the  overall  brightness  of  the 
CRT  displayed  information  would  be  diminished,  the  contrast  would 
be  improved . 


3.3  Mockup  Physical  Arrangement  (II) 

Problem:  Review  of  the  physical  configuration  of  the  IOS 

revealed  a  potential  problem  in  that  the  small  radius  of  the 
curved  work  surface  severely  limits  the  size  of  the  observer's 
viewing  space.  In  situations  where  additional  personnel,  whether 
operators  or  observers,  are  attempting  to  simultaneously  observe 
the  IOS  display  console,  unacceptable  crowding  is  probable.  This 
is  especially  likely  to  occur  during  transitions  between  simula¬ 
tor  training  sessions  when  one  instructor  and  student  are 
debriefing  while  a  new  instructor  and  student  await  an  opportu¬ 
nity  to  program  the  simulator  for  the  succeeding  lesson.  Expand¬ 
ing  the  size  of  the  IOS  may  be  unacceptable  from  the  standpoint 
of  operating  efficiency,  as  well  as  the  consequent  need  for 
additional  space  in  the  simulator  facility. 

Recommendation:  No  solution  to  this  problem  is  immediately 

apparent.  The  contractor  might  consider  other  form  factors  for 
the  IOS  which  provide  a  greater  viewing  volume  for  the  displays 
yet  do  not  increase  the  physical  spacing  between  them.  Clearly 
additional  study  is  required. 


8 


Note:  One  possibility  which  should  be  considered  is  a 

remote  debrief  or  lesson  programming  station  which  would  minimize 
simultaneous  demand  for  the  IOS.  This  suggestion  has  already 
been  discussed  in  detail.  (See  Section  2.2.) 


3.4  Instructor  Displays  of  Student  Controls 

Problem:  Though  present  F-16  OFT  plans  include  instructor 

displays  of  instrument  readings  and  major  control  settings, 
during  instrument  failure  for  the  student  it  is  also  planned  that 
the  instrument  reading  will  fail  also  for  the  instructor.  This 
will  render  the  instructor  incapable  of  determining  the  status  of 
the  student's  aircraft. 

Recommendation :  The  instructor  display  of  student  indica — 

tors  and  controls  should  show  appropriate  readings  continually 
rather  than  as  a  function  of  student  views . 


3 . 5  Page  Format 

Problem:  Simulator  CRT  pages  are  the  principal  means  of 

communication  of  both  situational  and  control  data  from  the 
simulator  to  the  instructor/operator.  As  potential  bottlenecks 
in  the  closed-loop  information  flow  between  machine  and  operator, 
the  capacity  of  the  CRT  pages  to  present  information  and  the 
capacity  of  the  operator  to  perceive  and  interpret  it  may  jointly 
limit  the  information  transmission  rate  around  the  loop.  The 
quantity  and  quality  of  information  exchanged  through  this  chan¬ 
nel  is  a  prime  measure  of  the  effectiveness  of  the  IOS  and  conse¬ 
quently  of  the  entire  OFT  as  a  training  tool.  All  too  frequently 
inadequate  attention  is  paid  to  structuring  information  on  CRT 
pages  in  a  way  that  makes  the  nature  of  the  information  clear  and 
the  exact  values  quickly  readable  and  easy  to  locate.  Also 
ignored  in  CRT  page  construction  are  the  perceptual  preferences 
of  the  user.  What  may  be  most  convenient  to  display  from  a  data 
processing  standpoint  may  not  be  the  most  interpretable  form. 

For  example,  easy  to  use  symbolic  or  pictorial  representations 
are  often  avoided  in  favor  of  easily  programmed  lists  and  tables. 

Recommendation:  The  F-16  OFT  IOS  display  will,  upon  demand, 

present  nearly  250  pages  of  data  to  the  instructor  for  his  use  in 
monitoring  and  controlling  simulator  exercises.  An  extensive 
study  of  instructor/operator  information  needs  and  problem  struc¬ 
tures,  combined  with  known  principles  of  efficient  display 
arrangement  would  no  doubt  lead  to  a  set  of  optimally-configured 
display  formats.  While  the  value  of  such  a  study  would  have  high 
payoff  in  increased  instructor  performance,  particularly  at  this 
stage  of  OFT  design,  the  necessary  resources  in  time  and  money 
are  not  available  to  the  F-16  ISD  effort. 

In  lieu  of  that  study  the  following  principles  and  examples 
are  offered  as  guidelines  in  the  design  of  F-16  OFT  CRT  page 


formats .  These  principles  have  been  found  through  empirical 
study  to  create  displays  which  promote  efficient  communication  of 
information  to  human  machine  operators.  The  organization  of 
information  for  interpretability  and  the  perceptual  requirements 
of  the  human  operators  have  been  taken  into  account  in  these 
principles . 

Principle  #1:  The  design  of  a  particular  page  must  be  made 
with  an  understanding  of  the  use  an  operator  will  make  of  the 
information  on  the  page.  On  the  P-16  IOS  display  CRT  pages  are 
used  For  three  principal  functions:  (1)  to  provide  feedback  on 
performance  of  the  student  on  a  simulated  task,  (2)  to  control 
the  task  structure  given  to  the  student,  and  (3)  to  control  the 
operational  state  of  the  simulator.  This  information  is  put  to 
various  uses  by  the  instructor.  Some  of  it  is  used  in  highly 
time-constrained  decisions  processes,  some  is  used  for  purposes 
which  require  a  high  degree  of  accuracy,  and  in  other  situations 
the  absolute  value  of  a  parameter  is  not  as  important  as  the 
direction  or  rate  of  change  of  a  parameter. 

Imagination  must  be  used  in  the  creation  of  display  formats 
appropriate  to  each  of  these  uses.  For  instance,  an  item  of 
information  which  must  be  quickly  read  with  moderate  precision 
may  be  shown  in  pictorial  form,  whereas  a  slowly  changing  para¬ 
meter  which  must  be  read  with  great  precision  must  be  presented 
in  the  form  of  a  numerical  read-out.  Numerical  readouts  are  not 
as  effective  as  indicators  of  rate  and  trend  information  nor  are 
pictorial  representations,  so  a  different  solution  must  be  found 
for  displaying  these  data.  A  thermometer-1 ike  analog  read-out 
could  show  both  momentary  value  and  rate  of  change  for  these 
data  . 


Principle  #2:  Information  displays  used  only  occasionally 
must  be  encoded  in  a  form  as  closely  as  possible  resembling  the 
observer's  natural  pictorial  and  verbal  language.  Any  device 
with  the  complexity  of  the  F-16  OFT  necessarily  imposes  a  heavy 
training  burden  on  its  operators.  Remembering  where  within  the 
250  CRT  pages  an  item  of  information  is  presented  may  present  an 
insurmountable  problem  to  the  casual  user.  The  problem  is  com¬ 
pounded  if  the  user  must  also  learn  non-symbol ic  codes  to  access 
or  change  data.  Research  on  display  design  has  demonstrated  that 
this  burden  can  be  significantly  lightened  by  maximizing  the 
correspondence  between  display  format  and  intuitive  expectation. 
The  use  of  English  phrases  and  pictorial  graphics  in  place  of 
abstract  codes  are  practices  which  promote  and  refresh  familiar¬ 
ity  with  information  identity.  Placing  information  in  geometric 
context  in  accordance  with  stereotypical  expectation  has  also 
been  found  to  help  reduce  access  time  and  promote  memory  for 
information  location.  The  application  of  these  principles  is 
difficult  to  generalize  but  is  largely  a  matter  of  common  sense. 

Principle  #3:  Where  quick  acquisition  of  displayed  informa- 
tion  is  reauired  by  the  demands  of  the  application,  the  amount  of 


information  on  the  display  must  be  limited  relative  to 


lute  capacity  for  presentation.  When  time  is  a  factor,  care 
should  be  exercised  to  take  advantage  of  decoding  and  formatting 
tricks  which  have  been  found  to  provide  more  rapid  access  to 
data.  Among  these  techniques  are  the  following: 

1.  Displays  should  be  simplified  to  contain  only  essential 

information.  It  is  very  difficult  to  extract  a 
particular  item  of  information  from  a  saturated 
display,  for  example,  to  find  a  number  in  a  column  or 
to  find  a  keyword  in  a  page  of  closely-packed  text. 

2.  Information  should  be  organized  into  related  chunks  so 
that  the  discrimination  task  becomes  one  of  searching 
for  a  class  of  items  rather  them  an  individual  item. 
These  classes  of  information  may  be  discriminated  from 
each  other  by  physical  location  encoding  or  by  setting 
off  an  entire  category  by  enclosing  it  in  a  box  (with  a 
label  at  the  head  of  the  box). 

3.  Redundant  coding  should  be  used.  The  indication  of 
critical  attention-demanding  items  by  flash-coding  or 
color-coding  and  the  indication  of  parameters  which  can 
be  changed  as  opposed  to  parameters  for  informational 
purposes  as  differences  in  type  face  or  display  inten¬ 
sity  or  denotative  symbols  such  as  underline. 

Principle  #4:  Organize  CRT  page  structure  along  logical 
1 ines .  When  in  any  hierarchical  system  of  CRT  pages  the  problem 
of  navigation  through  the  structure  of  the  page  arises, 
frustration  to  occasional  system  users  will  be  met.  Some 
suggestions  for  avoiding  losing  instructors/operators  in  the  page 
maze  follow: 

1.  Use  road  signs  and  road  maps.  Each  page  must  have  a 
title  which  conveys  its  logical  relationship  to  pages 
higher  and  lower  in  the  organizational  scheme. 

2.  Provide  access  to  orienting  information.  A  ’’Where  am  I?" 
function  which  calls  the  roadmap  to  the  display  with  a 
"you  are  here”  indication  and  information  on  how  to 
transfer  to  other  locations  is  recommended.  This  page 
should  be  accessible  to  the  user  by  a  single  control 
function.  Return  to  the  original  point  of  inquiry  should 
also  be  a  single  control  function. 

3.  Trade  breadth  for  depth.  CRT  page  structure  should  avoid 
being  many  pages  deep  in  favor  of  being  many  pages  wide. 

4.  Simplify  page  transition  from  any  given  page  to  any  other 
by  designating  the  title  of  the  destination  page.  While 
the  traditional  up-down  flow  through  hierarchy  of  pages 
is  useful  in  some  situations  and  should  be  preserved,  the 
direct  transition  capability  will  greatly  simplify  simu¬ 
lator  operation  and  reduce  the  memory  burden  for  occa¬ 
sional  operators. 
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Principle  #5:  The  psychophysical  properties  of  the  CRT 
display  should  be  appropriate  to  the  simulator  operating  environ¬ 
ment.  The  IOS  design  engineering  effort  roust  include  analytical 
consideration  of  the  contrast,  brightness,  and  surrounding 
ambient  illumination  level  for  the  IOS  display  to  insure  that 
adequate  viewability  of  the  displayed  information  is  obtained. 

In  addition,  factors  unique  to  specific  operating  environments 
must  be  considered  for  their  effect  on  display  interpretability. 

While  there  are  design  features  which  may  deal  with  distrac¬ 
ting  factors,  all  cannot  be  anticipated,  and  consideration  must 
be  given  to  the  selection  of  display  characteristics  which  mini¬ 
mize  the  impact  of  these  factors.  For  example,  the  use  of  color 
coding  or  flash  coding  is  of  questionable  value  in  an  environment 
filled  with  flashing  colored  lights  which  may  reflect  off  the 
screen. 


3.6  Automatic  Malfunctions  Indication 

Problem: 

1.  The  F-16  OFT  will  have  the  capability  to  introduce 
randomly  selected  malfunctions  to  the  simulated 
scenario,  either  as  part  of  an  automatic  malfunction 
generation  sequence,  or  as  a  consequence  of  simulated 
battle  damage. 

2.  When  malfunctions  are  manually  inserted  by  the 
instructor,  he  may  anticipate  the  upcoming  event  and  be 
prepared  to  note  appropriate  student  response  to  the 
situation. 

3.  When  these  malfunctions  appear  unannounced,  however,  the 
instructor  is  likely  to  be  as  surprised  as  his  student 
and  may  be  unable  to  monitor  the  student's  response  in 
the  desired  fashion. 

Recommendation:  The  F-16  simulator  should  provide  the 
instructor  with  advance  indications  o£  the  nature  and  time  of 
occurrence  of  automatically  inserted  malfunctions^  Wairning 
indications  should  appear  sufficiently  early  so  that  the 
instructor  can  take  action  to  modify  or  inhibit  the  scheduled 
event  if  desired.  Instructor  concurrence  with  automatically 
selected  events  is  not  required,  but  an  injunction  against  the 
automatic  action  should  be  easy  to  obtain  by  simple  control 
action  to  extend  the  instructor  response  time  available. 


3.7  Malfunction  Codes 

Problem:  Present  F-16  OFT  plans  call  for  the  selection  and 
indication  of  malfunction  options  using  numerical  codes.  These 
numeric  codes,  one  for  each  malfunction,  convey  no  semantic 
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information  about  the  real  name  of  the  malfunction  and  have  to  be 
memorized  by  instructors  by  rote.  Individual  instructors  using 
the  simulator  constantly  might  be  expected  to  memorize  these 
codes  to  the  level  of  usefulness.  In  common  experience,  however, 
individual  instructors  do  not  get  frequent  enough  exposure  to  the 
simulator  to  retain  an  adequate  degree  of  familiarity  with  such 
symbols.  As  a  consequence,  it  is  entirely  predictable  that  if  no 
better  device  for  symbolism  is  created  the  F-16  simulator  IOS 
will  be  wallpapered  with  crib-sheets.  The  constant  reference  to 
these  will  interrupt  the  smooth  and  efficient  flow  of  IOS  opera¬ 
tion.  Worse,  important  mission  events  may  escape  the 
instructor's  notice  while  he  searches  for  the  appropriate 
numerical  codes. 

Recommendation:  Simulator  designers  should  establish  a  list 
of  commonly  used  terms  and  denote  aircraft  system  events  with 
mnemonic  codes  for  those  terms.  Certainly,  mnemonic  codes  could 
be  used  to  indicate  active  malfuctions  on  non-interactive 
displays  with  very  little  design  impact.  Furthermore,  mnemonic 
codes  could  also  be  associated  with  numerical  codes  on  selection 
menus  to  clarify  instructor  options.  Optimally,  reference  to 
options  would  be  manipulated  by  instructors  using  these  mnemonics 
rather  than  meaningless  numerical  codes.  However,  it  is  recog¬ 
nized  that  major  IOS  design  impact  would  be  incurred  by  this 
arrangement.  Because  of  the  familiarity  and  meaningfulness  of 
these  mnemonic  terms  both  within  the  context  of  IOS  usage  and 
from  acquired  significance  through  common  usage,  instructors 
would  find  the  interpretation  of  IOS  displays  to  be  much  more 
quickly  learned  and  more  easily  retained. 


3.8  Catastrophic  Malfunction  Combinations 

Problem:  Certain  combinations  of  selectable  malfunctions 

may  preclude  successful  completion  of  the  simulated  mission 
either  by  elimination  of  essential  capabilities,  or  by  rendering 
the  simulated  aircraft  impossible  to  control.  While  under  calm 
circumstances  the  instructor  will  anticipate  the  consequences  of 
manually  inserted  malfunctions,  this  may  not  be  easily  possible 
when  the  instructor's  workload  is  increased  by  tactical  manage¬ 
ment  responsibilities,  communications,  performance  monitoring, 
flight  profile  amendment  actions,  etc.  In  such  stressful 
instances,  the  instructional  schedule  or  instructor  whim  may 
stimulate  the  insertion  of  an  individually  benign  but  combina- 
torially  catastrophic  malfunction. 

Recommendation:  Logic  should  be  built  into  the  IOS  software 
complement  to  provide  the  instructor  with  warning  that  emergency- 
combinations  will  be  counterproductive  to  the  objectives  of  the 
selected  mission.  While  the  logic  should  stop  snort  of  inhibi¬ 
ting  an  instructor  action,  advance  indication  of  possible  lethal 
combinations  of  selectable  events  should  be  provided.  At  a 
minimum,  a  challenge/confirmation  interchange  should  be  initiated 
by  manual  selection  of  such  incompatible  combinations  of  events. 
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4.0  DESIGN  FOR  GROWTH 


Through  the  service  life  of  the  F-16,  changes  can  be 
expected  in  the  airframe  and  avionics  which  will  impact  the 
instructional  system  design  and  the  configuration  of  the  F-16 
OFT  weapons  system  trainer  (WST) .  In  addition,  during  the  life¬ 
time  of  the  simulator,  advances  in  the  technology  of  simulator 
instruction  will  also  suggest  changes  to  enhance  the  OFT's  effec¬ 
tiveness  as  an  instructional  tool.  The  simulator  design  must  be 
sufficiently  flexible  that,  as  these  changes  create  new  require¬ 
ments  for  simulation  capability,  they  can  be  met  with  minimum 
impact  on  the  system  hardware.  Such  flexibility  is  obtainable, 
but  only  if  set  out  as  a  design  objective  early  in  simulator 
development . 


4.1  Design  for  Expansion  (I) 

Problems  The  F-16  OFT  is  being  procured  with  a  number  of 
optiors  under  consideration  for  later  addition.  Among  these 
optionr  are  computer-generated  visual  scene  simulation,  perfor¬ 
mance  : easur ament  capability,  adaptive  training  logic,  and 
automated  training  capability.  These  additions  may  take  the  form 
of  inc:  •it.ents  to  the  simulator  software  ensemble  if  sufficient 
couj/Utdti.onal  reserve  is  available  within  the  basic  simulator 
system.  (NOTE:  It  is  a  common  temptation  to  size  computational 
capabilities  as  closely  as  possible  to  the  current  need,  ignoring 
likely  future  demands  for  additional  capability.  This  should  be 
avoided  in  the  F-16  simulator.) 


In  all  likelihood,  one  or  more  of  the  additional  features 
projected  for  the  F-16  simulator  will  require  additional  hardware 
to  be  added  to  the  basic  simulator,  and  will  in  addition  require 
the  transferral  or  exchange  of  a  large  volume  of  data  at  high 
speed  between  the  presently  planned  and  add-on  data  processing 
equipment.  Since  the  simulator  must  perform  in  real-time,  there 
can  be  no  tolerance  for  interrupts  to  service  these  data 
requests.  Some  means  other  than  disrupting  the  flow  of  the  main¬ 
line  simulation  software  must  be  found  to  provide  the  required 
data  intercommunication  capability. 

Recommendation:  A  dual-port  common  data  (pool  memory  should 

service  the  simulator  while  simultaneously  providing  access  to 
important  parameters  in  memory  for  add-on  functions,  if  such  a 
solution  is  not  possible,  care  will  have  to  be  taken  to  ensure 
that  sufficient  CPU  speed  is  available  to  conduct  the  primary 
simulation  function  while  simultaneously  servicing  interrupts 
associated  with  the  various  add-on  options.  While  such  a  design 
practice  might  seem  obvious,  provision  of  reserve  speed  will 
necessarily  involve  increased  cost.  Compromise  for  cost  savings 
purposes  must  not  be  made  at  the  risk  of  incompatibility  with 
intended  add-on  options. 
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4.2  Design  for  Expansion  (II) 


Problem:  Two  training  environments  will  be  employed  for 

instruction  in  skills  associated  with  air  combat  maneuvering 
(ACM)  operations  in  the  F-16.  One  of  these,  of  course,  is  the 
aircraft.  ACM  missions  will  be  practiced  under  relatively  uncon- 
trolled  conditions  in  restricted  airspace  reservations,  and  under 
close  scrutiny  but  still  relatively  uncontrolled  conditions 
within  the  confines  of  AC MR  instrumented  facilities.  The  latter 
are  capable  of  recording  key  details  of  missions  for  later  replay 
and  review.  The  second  training  environment  for  the  introduction 
of  ACM  skills  is  scheduled  to  be  the  fully  expanded  WST  version 
of  the  F-16  flight  simulator.  There,  conditions  can  be  highly 
controlled,  but  will  be  of  questionable  realism  due  to  unavoid¬ 
able  fidelity  shortfalls.  It  may  be  highly  advantageous  to 
devise  a  mixed  media  approach  to  instruction  in  ACM  skills  that 
exploits  the  realism  of  the  AC MR  environment  while  capitalizing 
upon  the  control  afforded  by  the  F-16  WST.  One  such  strategy 
that  might  be  considered  would  be  to  use  the  WST  to  review  and 
repeat  AC MR  flown  exercises  in  the  simulator. 

Recommendation:  Provide  software  in  the  F-16  OFT/WST  to 

interpret  and  reconstruct  missions  flown  on  the  AC MR  using  rang e 
ta^es  produced  by  Cubic  Corporation  equipment  as  input  media. 

This  capability  should  enable  the  instructor  to  stop  action  or 
designate  any  point  in  the  AC MR  record  as  an  initial  condition 
set  for  beginning  an  autonomous  exercise  in  the  OFT/WST. 
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